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ABSTRACT 

Nova  Sensors  produces  miniature  electronics  for  a  variety  of  real-time  digital  video  camera  systems,  including 
foveal  sensors  based  on  Nova’s  Variable  Acuity  Superpixel  Imager  (VASI™)  technology.  An  advanced  image- 
processing  package  has  been  designed  at  Nova  Sensors  to  re-configure  the  FPGA-based  co-processor  board  for 
numerous  applications  including  motion  detection,  optical,  background  velocimetry  and  target  tracking. 
Currently,  the  processing  package  consists  of  14  processing  operations  that  cover  a  broad  range  of  point-  and 
area-applied  algorithms.  Flexible  FPGA  designs  of  these  operations  and  re-programmability  of  the  processing 
board  allows  for  easy  updates  of  the  VASI™  sensors,  and  for  low-cost  customization  of  VASI™  sensors  taking 
into  account  specific  customer  requirements. 

This  paper  describes  the  image  processing  algorithms  implemented  and  verified  in  Xilinx  FPGAs  and  provides 
the  major  technical  performances  with  figures  illustrating  practical  applications  of  the  processing  package. 

Keywords:  Foveal  Sensors,  Image  Processor,  FPGA,  FPA,  Variable  Acuity,  Superpixels,  Programmable, 
Visible,  Infrared,  UAV 


1.  INTRODUCTION 

A  new  class  of  imaging  sensors,  “foveal  sensors”,  has  been  developed  at  Nova  Sensors  for  the  purpose  of 
providing  a  wide  field  of  view  at  high  frame  rates  with  one  or  more  very  high  spatial  resolution  regions  on  areas 
of  interest.  These  sensors  are  based  on  Nova’s  Variable  Acuity  Superpixel  Imager  (VASI™)  technology  [1]. 
The  VASI™  devices  provide  the  capability  for  the  user  or  controlling  processor  to  change  the  spatial 
configuration  of  high-  and  lower  spatial  resolution  pixels  at  the  frame  rate.  The  multi-resolution  characteristics 
of  this  sensor  help  to  reduce  the  total  number  of  effective  pixel  values  that  are  multiplexed  off  of  the  focal  plane 
array  (FPA)  with  each  frame,  thereby  reducing  the  data  bandwidth  required  to  produce  an  accurate 
representation  of  the  salient  portions  of  the  image,  while  continuing  to  observe  the  total  field  of  view  (TFOV)  at 
lower  resolutions. 

Designed  according  to  principles  of  functioning  of  human  vision,  these  new  imaging  sensors  will  enable  a  large 
new  class  of  real-time  algorithms  specifically  designed  for  autonomous  target  acquisition  (ATA)  and  tracking) 
targets  of  interest.  It  is  increasingly  important  to  exploit  the  use  of  this  class  of  imaging  sensor  with  new 
algorithms  that  are  specifically  designed  to  make  use  of  the  spatially  tunable  nature  of  the  FPA.  To  date,  Nova 
Sensors  has  produced  a  variety  of  “first-cut”  acquisition  and  tracking  algorithms  for  the  VASI™.  These 
algorithms  are  being  implemented  into  Nova  Sensors’  miniature  image  processor  board  that  has  been  introduced 
in  [2].  A  brief  description  of  the  image  processor  is  given  in  Section  2. 

The  goal  of  this  paper  is  to  present  the  advanced  image-processing  package  for  field  programmable  gate  arrays 
(FPGA)  based  re-programmable  miniature  electronics  (see  Section  3).  The  basis  of  the  processing  package  is 
VASI™  pre-processor  and  multiple  target  acquisition  discussed  in  Section  3.1  and  Section  3.2,  correspondingly. 
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The  VASI™  pre-processor  includes  Image  Reconstruction,  one  and  two  point  Non-Uniformity  Correction 
(NUC),  NUC  controller,  Bad  Pixel  Replacement  (BPR),  and  2-bit  VASI™  FPA  controller  (see  Figure  2).  A  set 
of  14  other  processing  operations  that  could  be  implemented  as  a  parallel  or  sequential  FPGA-Processor  to 
support  a  consumer  requested  mission  are  presented  in  Section  3.3  and  listed  in  Table  1.  A  potential  candidate 
for  further  FPGA  implementation  is  morphological  target  acquisition  discussed  in  Section  4. 


Table  1.  Point-  and  area-applied  algorithms  included  in  the  advanced  image-processing  package 


Algorithms  Implemented  in  FPGA 

Master  Card 
Sections  3.1  and 
Section  3.2 

Parallel 
Processor 
Section  3.3 

Point-applied  algorithms: 

-  Image  reconstruction  from  a  compressed  frame 

* 

-  One-  and  two-point  non-uniformity  correction 
including  the  median  offset  computation  from  the 
masked  FPA-row  data 

* 

-  Arbitrary  pixel  revectoring 

* 

-  Bad  pixel  replacement 

* 

-  Pixel-based  mean  and  standard  deviation  computation 

-  Spectral  attenuation  compensation 

* 

-  Atmospheric  attenuation  compensation 

* 

Area-applied  algorithms: 

-  Intensity-weighted  centroid  target  tracking  (up  to 

* 

four  targets) 

* 

-  Real-time  Fast  Fourier  Transform  (FFT)  and  Power 
Spectral  Density  (PSD)  computation 

* 

-  Convolution  processing 

* 

-  Median  and  average  filters 

* 

-  Optical  flow  vector  field  computation 

* 

-  Background  velocimetry 

* 

-  Zero  Crossing  Technique 

Further  FPGA  implementation 

-  Morphologically-based  Target  Acquisition 

* 

2.  MINIATURE  IMAGE  PROCESSOR  BOARD 

Figure  1  shows  a  photograph  of  the  miniature  image  processor  board  set.  The  board  set  includes  a  number  of 
Xilinx  Virtex  II  FPGA  devices  (each  part  contains  at  least  one  million  gates  of  programmable  logic)  to  support  a 
wide  variety  of  processing  algorithm  operations  and  incorporates  enough  static  RAM  to  support  growth  to  image 
sizes  of  up  to  2048  x  2048  pixels  per  frame.  The  processor  board  set  is  capable  of  receiving  digital  camera 
image  data  at  effective  total  pixel  rates  of  up  to  80  megapixels/second.  Depending  upon  the  complexity  of  the 
processing  function,  multiple  boards  can  be  interfaced  to  partition  the  task  into  multiple  parallel  processing 
streams.  More  details  regarding  processor  board  performance  have  been  presented  in  [2]. 


Figure  1.  Nova’s  miniature  image/signal  processor  board  set,  designed  to  incorporate  multiple  Xilinx  Virtex  II 
FPGAs  and  fast  SRAM  memories. 

3.  ADVANCED  IMAGE  PROCESSING  PACKAGE 

The  advanced  image-processing  package  has  been  designed  at  Nova  Sensors  to  re-configure  the  FPGA-based  co¬ 
processor  board  for  numerous  applications  including  motion  detection  and  target  tracking.  Figure  2  shows  that 
the  basis  of  the  processing  package  is  the  VASI™  Pre-Processor  and  Multiple  Target  Acquisition  to  be  discussed 
in  Section  3.1  and  Section  3.2.  A  set  of  14  other  processing  operations  that  could  be  implemented  as  a  parallel 
or  sequential  FPGA-Processor  to  support  a  consumer  requested  mission  are  presented  in  Section  3.3.  The 
architecture  supports  expandability.  Additional  parallel  processor  boards  are  shown  as  dashed  lines.  These 
Miniature  Image  Processing  Cards  can  be  added  to  the  Master  Processing  Card  as  required. 


Figure  2.  Re-programmable  miniature  foveal  sensor  for  multiple  target  acquisition.  Additional  parallel 
processors  are  shown  as  dashed  lines. 


3.1.  VASI™  PRE-PROCESSOR 

3.1.1.  Up”-bit  and  “Left”-bit  Controller  to  program  FPA 

The  VASI™  Pre-Processor  firmware  design  performs  the  following  functions  on  the  sensor  data  stream:  “Up”-it 
and  “Left’ ’-bit  Controller  to  program  the  VASI™  FPA,  Image  Reconstruction,  Bad  Pixel  Replacement  (BPR), 
and  Non-Uniformity  Correction  (NUC,  see  Figure  2). 


“Up”  and  “Left”  denote  the  connectivity  between  an  individual  pixel  and  its  upward  and  leftward  nearest 
neighbors.  Each  pixel  is  programmed  into  a  specific  “superpixel”  state,  and  it  will  “know”  how  to  share  its 
charge  with  neighbors  so  that  a  resulting  superpixel  distribution  is  produced.  High  frame  rates  are  possible  using 
VASI™  technology  by  programming  the  device  into  a  superpixel  state;  pixels  (whether  they  be  “standard  pixels” 
or  “superpixels”)  need  to  deliver  only  one  analog  data  value  per  frame.  This  will  minimize  the  total  number  of 
pixel  values  to  be  read  off  of  the  FPA  (“compressed”  images  are  read  off  FPA),  thus  maximizing  effective  frame 
rate.  An  example  of  “Left”-  and  “Up”-  maps  is  presented  in  Figure  3  (see  images  b)  and  c),  respectively).  These 
maps  have  been  used,  correspondingly,  to  program  the  configuration  shown  in  the  image  d.  Four  dark  areas  in 
Figure  3  (images  b  and  c)  correspond  to  the  four  foveae.  Their  coordinates  and  dimensions  have  been  computed 
by  the  Multiple  Target  Acquisition  function  of  the  master  image-processing  card  (Figure  2).  In  the  given 
example  the  image  size  is  128x128  pixels  while  the  superpixel  size  is  16x16  pixels. 


3.1.2.  Image  Reconstruction 

As  it  has  been  shown  above  the  VASI™  technology  allows  for  “compressing”  images  (decreasing  the  number  of 
pixel  values)  read  from  the  FPA  while  maintaining  full  resolution  surrounding  areas  of  interest,  and  retaining 
detection  capability  over  the  entire  field-of-view  (FOV).  To  “uncompress”  images,  an  Image  Reconstruction 
function  is  performed  in  the  VASI™  Pre-Processor.  Figure  3  shows  both  a  full  resolution  image  (image  a)  and 
one  that  has  been  reconstructed  (image  d)  by  the  VASI™  Pre-Processor  after  the  FPA  has  been  programmed  into 
a  superpixel  configuration  with  two  foveal  regions.  Both  figures  contain  a  point-target  and  a  resolved  target. 
The  full  resolution  image  on  the  left  was  generated  by  the  CHAMP  infrared  scene  generation  program 
[3],  [4],  and  contains  16384  pixel  values  (128x128).  The  size  of  the  full  resolution  frame  has  been  reduced  from 
16384  pixel  values  to  718  pixel  values  in  the  “compressed”  superpixellated  image  on  the  right.  Each  superpixel 
is  16x16  in  size.  This  superpixel  configuration  with  two  foveal  regions  produces  23  times  less  pixel  values  per 
frame  than  the  full  resolution  image.  For  each  frame,  the  Image  Reconstruction  function  computes  the  standard 
deviation  and  mean  values  of  the  image  that  could  be  used  for  further  processing. 
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Figure  3.  Comparison  of  the  full  resolution  image  a)  and  foveal  image  d).  Both  figures  contain  a  point-target 
and  a  resolved  target  image.  The  size  of  the  full  resolution  frame  is  128x128  pixels.  The  superpixel  size  in  the 
right  figure  is  16x16  pixels.  The  b)  and  c)  images  represent  the  “Left”  and  “Up”  bits,  respectively,  used  to 
program  the  configuration  shown  in  the  image  d). 


3.1.3.  Two  Point  Spatial  Non-Uniformity  Correction  (NUC),  Bad  Pixel  Replacement  (BPR), 
NUC  Controller,  and  Arbitrary  Pixel  Revectoring 


Fixed  pattern  spatial  non-uniformities  in  the  CMOS  fabrication  process  used  to  fabricate  readout  integrated 
circuits  (ROIC)  cause  variations  in  the  gain  and  offset  characteristics  of  amplifiers  within  individual  ROIC  unit 
cells  or  columns  (see  Figure  4. a).  To  fix  this  problem  (see  Figure  4.b),  the  spatial  “two-point”  non-uniformity 
correction  is  included  in  the  VASI™  Pre-Processor,  i.e. 

Si  J  =  G)j  xj  .  +  C)j  for  i  =  1,  nx  and  j  =  1,  ny  (1) 

where  Xy  and  Sy  are  the  image  values  before  and  after  the  spatial  NUC  is  applied.  Coefficients  G1^  and  C^j  are 
gain  and  offset  respectively  that  fix  the  spatial  non-uniformities  and  nx  and  ny  are  the  total  number  of  pixels 
along  the  corresponding  axis,  X  and  Y  (see  also  Figure  7). 

Two-point  NUC  is  typically  performed  so  that,  assuming  the  linear  behavior  of  all  pixels  over  a  specified  flux 
range,  all  pixels  will  exhibit  the  same  signal  swing  response.  If  a  few  columns  in  FPA  sensors  are  protected 
from  illumination,  the  offset  Cl[  j  can  be  computed  in  the  FPGA  as  means  and  averages  from  the  above- 
mentioned  corresponding  columns.  As  a  result  of  high  temporal  noise,  low  responsivity,  etc.,  some  pixels  can  be 
classified  as  “bad”  pixels.  The  VASI™  Pre-Processor  automatically  replaces  these  pixels  with  values  of  “good” 
spatial  neighbors  such  that  a  ’’nice-looking  picture”  is  produced  by  the  FPA.  The  NUC  Controller  applies 
corrections  to  pixel  values  within  high  resolution  (i.e.  fovea)  areas  only.  In  order  to  keep  the  effective  frame  rate 
of  a  large  pixel-count  imager  high,  FPAs  typically  multiplex  parallel  channels  of  data  at  the  output.  If  such  an 
FPA  is  used,  the  VASI™  Pre-Processor  “shuffles”  the  parallel  channels  of  data  together  to  re-assemble  a 
properly  displayed  image.  The  so-called  Arbitrary  Pixel  Revectoring  (APR)  provides  arbitrary  spatial  re¬ 
mapping  of  these  pixel  values  by  accessing  re-vector  address  memory  that  stores  the  appropriate  re-vectored 
locations. 


Figure  4.  Full  resolution  infrared  image  before  NUC  and  BPR  (the  left  figure),  and  the  corresponding  image 
after  the  spatial  NUC  and  BPR  (the  right  figure). 


3.2.  AUTONOMOUS  MULTIPLE  TARGET  ACQUSITION  (AT A) 

An  Autonomous  Multiple  Target  Acquisition  technique  for  foveal  sensors  such  as  VASI™  has  also  been 
developed.  The  current  FPGA  implementation  of  this  technique  allows  for  acquiring  up  to  four  targets 
simultaneously.  Figure  3  (images  c,  b,  and  d)  illustrates  a  specific  example  when  four  foveae  have  been 
allocated  by  the  AT  A  algorithm  around  a  relatively  large  image  of  a  resolved  target  and  a  single  point  target. 
Nova’s  Multiple  Target  Acquisition  technique  does  not  use  thresholds  to  detect  targets  of  interest,  but  instead 
uses  an  area-  and  intensity-  computation  technique.  Using  this  technique,  the  size  of  each  rectangular  fovea 
depends  on  the  shape  and  dimension  of  the  image  area  selected  as  a  most  competitive  target  candidate  (see 
Figure  5  and  Figure  6).  The  left  frame  in  Figure  5  shows  a  full  resolution  image.  When  the  AT  A  algorithm  is 
applied,  the  FPA  is  configured  into  a  superpixel  mode  with  three  fovea  located  over  detected  objects.  This  can 
be  seen  in  the  reconstructed  image  on  the  right. 


a).  Full  Resolution  Image 


b).  Fovea  Image 


Figure  5.  Comparison  of  a  full  resolution  image  (left  figure)  and  foveae  image  (right  figure).  The  size  of  the  full 
resolution  frame  is  128x128  pixels.  The  superpixel  size  in  the  right  image  is  16x16  pixels.  The  size  of  image  on 
the  right  is  about  23  times  smaller  than  the  full  resolution  image  on  the  left.  The  corresponding  “Up”  and  “Left” 
bits  Maps  are  shown  in  Figure  6. 


Figure  6.  The  left  and  right  images  above  represent  the  “Up”  and  “Left”  bits,  respectively,  used  to  program  the 
FPA  configuration  shown  in  Figure  5.b.  The  FPA  format  is  128x128  pixels.  The  superpixel  size  is  16x16  pixels. 
Three  dark  areas  correspond  to  the  three  fovea  whose  coordinates  and  dimension  have  been  computed  by  the 
VASI™  Pre-Processor  (see  Figure  2). 

3.3.  ADDITIONAL  PARALLEL  PROCESSOR 

A  list  of  point-  and  area-applied  algorithms  implemented  in  the  Image  Processor’s  FPGA  is  presented  in  Table  1. 
The  second  column  of  the  table  indicates  algorithms  implemented  in  the  Master  Card  (see  Figure  2).  Additional 
parallel  processors  (shown  in  Figure  2)  could  include  the  rest  of  the  point-applied  and  area-applied  algorithms 
listed  in  the  third  column  in  Table  1.  The  well-known  convolution  operations  as  well  as  median  and  averaging 
filters  implemented  in  an  FPGA  are  not  discussed  in  this  paper. 

3.3.1.  Atmospheric  and  Spectral  Attenuation  Correction 

Figure  7  shows  the  combined  spatial  non-uniform,  and  spectral-  and  atmospheric  attenuation  corrections 
implemented  in  Xilinx  FPGA  under  the  Low  Earth-Orbiting  Imaging  Sensor  (LEOIS)  program  (sponsored  by 
AFRL/VSSS).  Spatial  two-point  NUC  (see  Eq.  (1))  has  been  discussed  in  Section  3.1.2.  Signal  variations  due  to 
the  selective  absorption  of  radiant  energy,  yi  J?  as  a  function  of  wavelength  across  the  spectral  range  of  the 
hyperspectral  sensor  may  be  compensated  using  one-point  spectral  correction: 

s’j  =GSjj  su  ,  for  i  =  1,  nx  and  j  =  1,  ny 
where  pre-computed  gain  corrections,  Gs ;  j,  are  stored  in  SRAM. 


(2) 


Y  ij 

They  are  clocked  out  of  memory  and  applied  synchronously  with  the  pixel  data  stream, 
compensates  the  atmospheric  attenuation  by  the  one-point  correction 

siaj  =Gij  siSj  >  f°r  *  =  r  nx  and  j  =  1,  ny 
where  atmospheric  attenuation  gain,  Gaij,  is  computed  by 

G“,  =  — ,  for  i  =  1,  nx  and  j  =  1,  n 

t. 


Atmospheric  NUC 

(4) 

(5) 


and  ‘li’  is  the  atmospheric  transmittance  shown  in  Figure  8  for  the  middle-  (MWIR,  the  left  figure)  and  the  long- 
(LWIR,  the  right  figure)  infrared  wavelength  bandwidths. 


In  the  IP  FPGA  hardware  developed  at  Nova,  separate  address  ranges  in  memory  are  programmed  with 
atmospheric  attenuation  gain  data,  Ga  i  j,  and  spectral  correction  gains,  GS  j  j. 
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Figure  7.  Spatial,  spectral  and  atmospheric  corrections. 


a).  MWIR  b).  LWIR 


Figure  8.  Atmosphere  transmittance  in  MWIR  (the  left  figure)  and  in  LWIR  (the  right  figure)  measured  for  a 
dry  atmosphere  from  Earth  to  Space. 


3.3.2.  Real-time  Pixel-Based  FFT  and  PSD  Computation  for  Target  Acquisition 


256-  and  1024-point  Fast  Fourier  Transform  (FFT)  modules  have  been  designed  by  Nova  Sensors  using  fixed- 
point  arithmetic.  The  FFT  design  has  been  implemented  in  the  Xilinx  Virtex  II  family  of  field  programmable 
gate  arrays.  This  FFT  module  computes  real-time  power  spectra  on  the  VASI™  camera  data  stream. 

Figure  9  illustrates  one  of  the  practical  FFT-module  applications  in  the  “hyper-temporal”  processing  algorithm 
set  for  evaluation  of  temporal  signatures  of  objects  that  emit  modulated  IR  energy  in  the  audio  regime.  In  such 
an  application,  the  FFT  module  performs  real-time  evaluations  of  power  spectral  densities  (PSDs)  for  each  pixel 
in  the  imager,  followed  by  conditional  tests  to  attempt  “to  find”  single-pixel  objects  with  such  temporal 
signatures.  More  details  regarding  the  FFT  module  design  and  its  implementation  for  “hyper-temporal” 
processing  have  been  published  in  [5]. 


FFT-Based  Target  Acquisition 


Target  Coordinates 
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Figure  9.  Simplified  block  diagram  of  the  FFT-based  Target  Acquisition.  Nova’s  Virtex  II-based  PSD  processor 
permits  dynamically-adjustable  parameters  to  be  set:  nblocks  is  the  number  of  the  coadded  FFT  transforms; 
{ny*nx}  is  the  frame  size;  nfft  is  the  number  of  FFT  points,  i.e.  256  or  1024. 

3.3.3.  Optical  Flow  Vector  Field  Computation 

Other  applications  exist  that  require  a  real-time  estimation  of  the  image  velocity  flow  field  (see  Figure  10).  This 
may  be  applied  to  sensor  egomotion  sensing,  object  identification  and  tracking  against  a  moving  background. 


Figure  10.  Background  velocimetry  using  Optical  Flow:  The  original  IR  image  shown  in  the  left  frame 
indicates  motion  of  the  sensor  from  left  to  right.  The  Optical  flow  field  shown  in  the  right  figure  estimated  the 
background  velocity  of  {-0.5  and  0.074}  in  X-  Y-directions,  respectively,  in  relative  numbers. 


In  addition,  these  techniques  can  be  used  to  compute  focus  of  expansion,  time  to  contact  and  many  other  motion 
properties  of  images  that  can  be  used  to  control  robots  or  other  moving  platforms.  Tracking  systems  can  use  the 
extracted  velocity  information,  while  segmentation  of  moving  objects  can  be  performed  by  using  the  detection  of 
motion  discontinuities  [6].  Using  optical  flow  fields,  spatial  clutter  may  be  reduced  by  eliminating  pixel 
responses  moving  in  specified  directions.  A  “first-cut”  Virtex  II  design  for  an  optical  flow  computation  engine 
based  on  principals  developed  by  Horn  and  Schunk  [7]  has  been  designed.  An  example  of  an  application  of 
optical  flow  for  background  velocity  measurement  is  shown  in  Figure  10.  The  original  IR  image  shown  in  the 
left  figure  indicates  motion  of  a  space-based  IR  sensor  from  West  to  East.  The  optical  flow  field  shown  in  the 
right  figure  estimated  the  background  velocity  of  {-0.5  and  0.074,  relative  numbers}  in  X-  Y-directions, 
correspondingly.  It  shows  that  the  optical  flow  technique  confirms  that  the  sensor  moves  from  left  to  right. 

3.3.4.  Background  Velocimetry 

We  have  developed  an  alternative  technique  for  background  velocimetry  based  on  the  computation  of  time-  and 
spatial-  derivatives  for  each  of  four  bars  A-,  B-,  C,  and  D-  arranged  along  the  corresponding  borders  of  an  FPA 
image  (see  Figure  11).  This  technique  may  use  Nova’s  VASI™  sensors  to  keep  frame  rates  high,  but  it  may  also 
be  applied  in  general  to  common  image  frame  sequences.  An  example  of  the  background  velocity  measurement 
is  shown  in  Figure  12. 
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Figure  11.  Background  velocimetry:  time-  and  spatial  derivatives  are  computed  for  each  of  four  bars  A-,  B-,  C, 
and  D-  bars  allocated  virtually  along  the  corresponding  borders  of  FPA. 


Figure  12.  The  background-motion  sensor  response  (the  right  figure)  from  the  non-periodic  background 
accelerated  in  horizontal  direction  (the  left  figure). 


The  background-motion  sensor  response  is  presented  in  the  right  figure  for  the  non-periodic  background 
accelerated  in  horizontal  direction  in  the  left  figure.  Zero  velocities  measured  at  the  A-  and  C-bars  (marked  by 
squares  and  stars,  respectively)  confirm  the  background  motion  along  the  X-axis.  The  linear  sensor  response  at 
B-  and  D-  bars  (see  the  left  figure,  lines  B  and  D,  marked  by  circles  and  triangular,  respectively)  is  in  agreement 
with  linear  acceleration  of  the  background. 


3.3.4.  Zero  Crossing  Technique 

Zero  Crossing  Technique  (ZCT)  includes  three  major  processing  steps:  blurring  of  each  frame,  computing 
Difference-of-Gaussian  (DoG),  and  zero  crossing  (extracting  pixels  with  intensities  close  to  zero  at  the  DoG 
output).  ZCT  can  be  used  for  edge  detection  and  as  a  preprocessing  step  for  Differential  Velocity  Detection 
developed  at  Nova  Sensors,  as  a  stay-along  target  tracker,  and  as  a  preprocessing  operation  for  a  more 
sophisticated  target  tracker  operating  in  the  midst  of  cluttered  background.  The  use  of  the  Zero  Crossing 
Technique  for  target  acquisition  is  demonstrated  in  Figure  13.  The  left  figure  is  the  original  image  that  contains 
a  point-target  and  an  air-target  image.  The  right  image  is  the  ZCT’s  output. 


Figure  13.  Use  of  Zero  Crossing  Technique  for  target  acquisition  and  for  edge  detection.  The  left  figure  is  the 
original  image  that  contains  a  point-target  and  an  air-target  image  [3],  [4].  The  right  image  is  the  ZCT’s  output. 


4.  FURTHER  FPGA  IMPLEMENTATION:  MORPHOLOGICALLY-BASED  TARGET 

ACQUISITION 

Our  further  work  includes  implementation  of  Morphologically-based  Target  Acquisition  (MTA)  techniques  into 
FPGA  implementations.  MTA  is  based  on  morphological  operations,  i.,e.,  dilation,  erosion,  opening,  and  closing 
[9], [10].  Figure  14  illustrates  a  block  diagram  of  MTA  developed  at  Nova  Sensors  using  an  adaptive 
computation  of  the  threshold  by  Renyi’s  entropy  [11].  Figure  15  demonstrates  competitive  target- acquisition 
performances.  The  left  figure  shows  an  IR  image  with  a  cluttered  background  moving  from  the  left  to  the  right 
(see  also  Figure  10),  and  a  synthetically  inserted  point-target  moving  from  the  top  to  the  bottom.  The  right 
figure  is  the  accumulative  output  from  the  Morphological  Filter  that  rejects  the  cluttered  background  and 
acquires  the  point-target.  The  right  figure  plots  the  point-target  trajectory  during  a  set  of  frames.  No  false 
alarms  were  computed. 


Adaptive  Morphologically-Based  Target  Acquisition: 
Renyi’s  Entropy  Threshold 
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Figure  14.  Block  diagram  of  Morphological  Target  Acquisition  (MTA)  with  adaptive  computation  of  threshold 
using  Renyi’s  entropy. 


a). 


Figure  15.  The  left  figure  illustrates  an  IR  image  with  a  cluttered  background  moving  from  right  to  left  (see  also 
Figure  10)  and  with  a  point-target  moving  from  the  top  to  bottom.  The  right  figure  is  the  accumulative  output 
from  the  Morphological  Filter  that  rejects  the  cluttered  background  and  acquires  the  point-target. 


5.  CONCLUSIONS 

All  of  the  image-processing  algorithms  listed  in  Table  1  have  been  developed  for  VASI™  sensors  and  have  been 
implemented  in  the  miniature  FPGA-based  processing  board  (see  Figure  1)  developed  by  Nova  Sensors.  The 
major  performances  of  the  each  technique  as  well  as  examples  of  their  practical  applications  were  discussed  in 
Section  2  and  Section  3.  Flexible  FPGA  designs  of  these  techniques  and  re-programmability  of  the  processing 
board  allows  for  low-cost  and  quick  turn-around  of  customized  applications  of  VASI™  and  other  state  of  the  art 
imaging  sensors,  taking  into  account  specific  customer  requirements. 
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7.  ABBREVIATIONS 


AT  A  -  autonomous  target  acquisition 

ATT  -  autonomous  tracking  targets  of  interest 

BPR  -  bad  pixel  replacement 

FFT  -  Fast  Fourier  Transform 

FPGA  -  Field  Programmable  Gate  Arrays 

NUC  -  non-uniformity  correction 

PSD  -  Power  Spectral  Density 

TFOV  -  total  field  of  view 

VASI™  -  Variable  Acuity  Superpixel  Imager  (Nova  Sensor’s  Trade  Mark) 
ZCT  -  zero  crossing  technique 
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